ABSTRACT
Introduction Rapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid evaluations, ensure scientific rigour, and achieve stakeholder engagement within compressed timeframes. Methods Using a case study of a national mixed-methods rapid evaluation of COVID-19 remote home monitoring services in England, conducted during the COVID-19 pandemic, this manuscript examines the process of conducting a large-scale rapid evaluation from design to dissemination and impact, and reflects on the key lessons for conducting future large-scale rapid evaluations. In this manuscript, we describe each stage of the rapid evaluation: convening the team (study team and external collaborators), design and planning (scoping, designing protocols, study set up), data collection and analysis, and dissemination. Results We reflect on why certain decisions were made and highlight facilitators and challenges. The manuscript concludes with 12 key lessons for conducting large-scale mixed-methods rapid evaluations of healthcare services. We propose that rapid study teams need to: (1) find ways of quickly building trust with external stakeholders, including evidence-users;(2) consider the needs of the rapid evaluation and resources needed;(3) use scoping to ensure the study is highly focused;(4) carefully consider what cannot be completed within a designated timeframe;(5) use structured processes to ensure consistency and rigour;(6) be flexible and responsive to changing needs and circumstances;(7) consider the risks associated with new data collection approaches of quantitative data (and their usability);(8) consider whether it is possible to use aggregated quantitative data, and what that would mean when presenting results, (9) consider using structured processes & layered analysis approaches to rapidly synthesise qualitative findings, (10) consider the balance between speed and the size and skills of the team, (11) ensure all team members know roles and responsibilities and can communicate quickly and clearly;and (12) consider how best to share findings, in discussion with evidence-users, for rapid understanding and use. Conclusion These 12 lessons can be used to inform the development and conduct of future rapid evaluations in a range of contexts and settings.
ABSTRACT
Introduction: Rapid evaluations can offer evidence on innovations in health and social care that can be used to inform fast-moving policy and practise, and support their scale-up according to previous research. However, there are few comprehensive accounts of how to plan and conduct large-scale rapid evaluations, ensure scientific rigour, and achieve stakeholder engagement within compressed timeframes. Methods: Using a case study of a national mixed-methods rapid evaluation of COVID-19 remote home monitoring services in England, conducted during the COVID-19 pandemic, this manuscript examines the process of conducting a large-scale rapid evaluation from design to dissemination and impact, and reflects on the key lessons for conducting future large-scale rapid evaluations. In this manuscript, we describe each stage of the rapid evaluation: convening the team (study team and external collaborators), design and planning (scoping, designing protocols, study set up), data collection and analysis, and dissemination. Results: We reflect on why certain decisions were made and highlight facilitators and challenges. The manuscript concludes with 12 key lessons for conducting large-scale mixed-methods rapid evaluations of healthcare services. We propose that rapid study teams need to: (1) find ways of quickly building trust with external stakeholders, including evidence-users; (2) consider the needs of the rapid evaluation and resources needed; (3) use scoping to ensure the study is highly focused; (4) carefully consider what cannot be completed within a designated timeframe; (5) use structured processes to ensure consistency and rigour; (6) be flexible and responsive to changing needs and circumstances; (7) consider the risks associated with new data collection approaches of quantitative data (and their usability); (8) consider whether it is possible to use aggregated quantitative data, and what that would mean when presenting results, (9) consider using structured processes & layered analysis approaches to rapidly synthesise qualitative findings, (10) consider the balance between speed and the size and skills of the team, (11) ensure all team members know roles and responsibilities and can communicate quickly and clearly; and (12) consider how best to share findings, in discussion with evidence-users, for rapid understanding and use. Conclusion: These 12 lessons can be used to inform the development and conduct of future rapid evaluations in a range of contexts and settings.
ABSTRACT
Background: There was a national roll out of 'COVID Virtual Wards' (CVW) during England's second COVID-19 wave (Autumn 2020 - Spring 2021). These services used remote pulse oximetry monitoring for COVID-19 patients following discharge from hospital. A key aim was to enable rapid detection of patient deterioration. It was anticipated that the services would support early discharge, reducing pressure on beds. This study is an evaluation of the impact of the CVW services on hospital activity. Methods: Using retrospective patient-level hospital admissions data, we built multivariate models to analyze the relationship between the implementation of CVW services and hospital activity outcomes: length of COVID-19 related stays and subsequent COVID-19 readmissions within 28 days. We used data from more than 98% of recorded COVID-19 hospital stays in England, where the patient was discharged alive between mid-August 2020 and late February 2021. Findings: We found a longer length of stay for COVID-19 patients discharged from hospitals where a CVW was available, when compared to patients discharged from hospitals where there was no CVW (adjusted IRR 1·05, 95% CI 1·01 to 1·09). We found no evidence of a relationship between the availability of CVW and subsequent rates of readmission for COVID-19 (adjusted OR 0.97, 95% CI 0.91 to 1·03). Interpretation: We found no evidence of early discharges or changes in readmissions associated with the roll out of COVID Virtual Wards across England. Our analysis made pragmatic use of national-scale hospital data, but it is possible that a lack of specific data (for example, on which patients were enrolled and on potentially important confounders) may have meant that true impacts, especially at a local level, were not ultimately discernible. It is important that future research is able to make use of better quality - preferably linked - data, from multiple sites. Funding: This is independent research funded by the National Institute for Health Research, Health Services & Delivery Research program (RSET Project no. 16/138/17; BRACE Project no. 16/138/31) and NHSE&I. NJF is an NIHR Senior Investigator.
ABSTRACT
INTRODUCTION: Remote home monitoring models were implemented during the COVID-19 pandemic to shorten hospital length of stay, reduce unnecessary hospital admission, readmission and infection and appropriately escalate care. Within these models, patients are asked to take and record readings and escalate care if advised. There is limited evidence on how patients and carers experience these services. This study aimed to evaluate patient experiences of, and engagement with, remote home monitoring models for COVID-19. METHODS: A rapid mixed-methods study was carried out in England (conducted from March to June 2021). We remotely conducted a cross-sectional survey and semi-structured interviews with patients and carers. Interview findings were summarized using rapid assessment procedures sheets and data were grouped into themes (using thematic analysis). Survey data were analysed using descriptive statistics. RESULTS: We received 1069 surveys (18% response rate) and conducted interviews with patients (n = 59) or their carers (n = 3). 'Care' relied on support from staff members and family/friends. Patients and carers reported positive experiences and felt that the service and human contact reassured them and was easy to engage with. Yet, some patients and carers identified problems with engagement (e.g., hesitancy to self-escalate care). Engagement was influenced by patient factors such as health and knowledge, support from family/friends and staff, availability and ease of use of informational and material resources (e.g., equipment) and service factors. CONCLUSION: Remote home monitoring models place responsibility on patients to self-manage symptoms in partnership with staff; yet, many patients required support and preferred human contact (especially for identifying problems). Caring burden and experiences of those living alone and barriers to engagement should be considered when designing and implementing remote home monitoring services. PATIENT OR PUBLIC CONTRIBUTION: The study team met with service users and public members of the evaluation teams throughout the project in a series of workshops. Workshops informed study design, data collection tools and data interpretation and were conducted to also discuss study dissemination. Public patient involvement (PPI) members helped to pilot patient surveys and interview guides with the research team. Some members of the public also piloted the patient survey. Members of the PPI group were given the opportunity to comment on the manuscript, and the manuscript was amended accordingly.